Papers with WhoQA questions
Who’s Who: Large Language Models Meet Knowledge Conflicts in Practice (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large-scale pretrained language models excel in tasks requiring natural language understanding, but they often "hallucinate" plausible but incorrect content due to outdated or incorrect pretraining information. |
| Approach: | They propose a public benchmark dataset to examine model’s behavior in knowledge conflict situations. |
| Outcome: | The proposed model induces conflicts by asking about a common property among entities having the same name, resulting in questions with up to 8 distinctive answers. |